Probing transfer learning with a model of synthetic correlated datasets

Transfer learning can significantly improve the sample efficiency of neural networks, by exploiting relatedness between a data-scarce target task and data-abundant source task. Despite years successful applications, transfer practice often relies on ad-hoc solutions, while theoretical understanding these procedures is still limited. In present work, we re-think solvable model synthetic data as framework for modeling correlation data-sets. This setup allows an analytic characterization generalization performance obtained when transferring learned feature map from to Focusing problem training two-layer networks in binary classification setting, show that our capture range salient features with real data. Moreover, parametric control over two data-sets, systematically investigate under which conditions beneficial generalization.